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1.  INTRODUCTION 


Background 

Distance  Learning  Project  North  began  in  September  1988  and  was  one  of  two 
developmental  projects  in  Alberta  with  a  primary  emphasis  on  increasing  equity  in  educational 
opportunity.  The  other  project  was  the  Distance  Learning  in  Small  Schools  Project  which  was 
concentrated  geographically  in  the  southeastern  quadrant  of  the  province.  In  contrast,  Project 
North  was  located  in  the  northwestern  quadrant  in  a  quadrilateral,  the  points  of  which  are 
Meander  River  to  the  north,  Slave  Lake  to  the  east,  Silver  Valley  to  the  west,  and  Fox  Creek  to 
the  south.  Documentation  provided  by  the  Project  Office^  describes  the  focus  of  Project  North 
as  being  to  test  altemative  distance  education  delivery  systems  which  can  be  used  to  provide 
equitable,  cost-effective  education  to  the  project  schools.  The  first  thrust  of  the  project  was  to 
explore  a  multi-grade  model  where  a  "classroom"  was  composed  of  students  from  a  number  of 
grades  in  a  common  subject  area.  The  same  document  indicates  that  the  plan  also  proposed  that 
a  multi-subject  approach  be  undertaken  in  which  students  from  a  number  of  subject  areas  and 
grade  levels  be  assembled  in  a  distance  learning  classroom.  Students  in  these  classes  would 
study  modular,  independent  learning  materials  while  receiving  instructional  support,  question 
and  answer  support,  evaluation  and  assessment  from  off-site.  A  third  component  was  to  have 
been  a  live,  interactive  French  program  delivered  via  satellite  to  selected  schools. 

The  project  was  coordinated  by  the  Alberta  Correspondence  School  and  a  steering 
committee  composed  of  senior  administrators  of  the  districts  involved,  together  with 
representatives  from  other  parts  of  Alberta  Education;  Digital  Equipment  Corporation  (DEC); 
Computer  Based  Training  Systems  (CBTS);  Technology,  Research  and  Telecommunications 
(TRT);  Alberta  Government  Telephones  (ACT);  North  Peace  Adult  Consortium;  and  Fairview 
College.  A  contract  was  let  to  Fairview  College  to  provide  a  coordinator  who  was  responsible 
for  hardware  and  software  installation,  inservice,  and  general  trouble- shooting.  The 
implementation  took  place  in  stages.  The  first  stage  implemented  in  Fall,  1988,  was  the  multi- 
grade  class  and  the  subject  area  selected  was  mathematics.  The  Alberta  Correspondence 


Table  LI 

Schools  participating  in  Year  One  of  Project  North. 


County  of  Grande  Prairie  #1 

Sexsmith  Secondary 
Beaverlodge 

East  Smoky  S.D.  #54 

Ridgevalley 
Fox  Creek 

Fairview  RCSSD  #35 

St.  Thomas  More  (Fairview) 

Fairview  SD  #50 

Hines  Creek 
Worsley 

Falher  Consolidated  S.D.  #69 

Ecole  Routhier 

rort  Vermilion  b.D.  #52 

High  Level 
La  Crete 
Rainbow  Lake 
Fort  Vermilion  Public 
Rocky  Lane 

Upper  Hay  River  (Meander  River) 

Grande  Prairie  RCSSD  #28 

St.  Joseph's  (Grande  Prairie) 

Grande  Prairie  S.D.  #2357 

Grande  Prairie  Composite 

High  Prairie  S.D.  #48 

Kinuso 

E.  W.  Pratt  (High  Prairie) 
Roland  Michener  (Slave  Lake) 

North  Peace  RCSSD  #43 

Glen  Mary  (Peace  River) 
Holy  Family  (Grimshaw) 

Peace  River  #10 

Paul  Rowe  (Manning) 
Grimshaw  Jr-Sr  High  School 

Spirit  River  S.D.  #47 

Eaglesham 
Savanna 
Spirit  River 
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School  revised  its  mathematics  courses  to  provide  new  independent  study  materials  for 
students  in  all  courses  in  mathematics.  Digital  Equipment  Corporation's  Micro  VAX  2000 
series  computers  were  acquired  for  participating  sites,  and  a  computer  managed  learning 
system  developed  and  distributed  by  Computer  Based  Training  Systems  was  installed  on  the 
Micro  VAX  2000s  to  handle  the  functions  of  student  tracking  and  testing. 

In  most  schools  which  implemented  CML  mathematics,  a  stand-alone  MicroVAX 
system  was  installed.  In  one  case  (Sexsmith  Secondary),  however,  a  MicroVAX  system  was 
installed  that  could  be  used  remotely,  and  Grand  Prairie  Composite  and  St.  Joseph's  in  Grande 
Prairie  installed  terminals,  printers,  and  modems  only,  and  used  these  to  access  the  system  in 
Sexsmith.  In  the  fall  of  1989,  Grande  Prairie  Composite  discontinued  taking  its  CML  feed 
from  Sexsmith  and  installed  its  own  system.  St.  Joseph's  is  continuing  to  take  its  feed  from 
Sexsmith,  and  Beaverlodge  is  planning  to  access  Sexsmith' s  system  beginning  after 
Christmas,  1989.  Similarly,  in  1988-89,  Holy  Family,  Grimshaw  took  its  feed  from  Glenmary 
but  since  then  has  purchased  its  own  VAX  system.  (Since  Summer,  1989,  Digital  has 
discontinued  making  the  MicroVAX  2000  series  and  now  offers  the  3000  series.) 

In  the  second  stage  of  the  project,  implemented  in  January  1989,  additional  equipment 
supportive  of  distance  delivery  was  acquired.  FAX  machines  were  acquired  for  all  schools. 
Most  acquired  teleconferencing  convenors,  and  some  acquired  audiographic  systems. 
Teleconferencing  convenors  permit  voice-only  communication  among  students  at  participating 
sites  which  is  commonly  referred  to  as  audioconferencing.  Audiographic  systems  include  the 
transmission  of  computer  graphics  as  well  as  voice.  The  project  provided  inservice  training  in 
the  operation  of  the  new  technology,  but  no  formal  plans  seem  to  have  been  prepared  for  its 
implementation  for  instructional  purposes.  Several  schools  did  try  out  audioconferencing  in 
both  instructional  and  non-instructional  contexts.  A  number  of  obstacles  arose  which 
prevented  completion  of  the  satelHte  French  component  of  the  project,  and  this  was  dropped. 
A  plan  to  produce  a  series  of  mathematics  vignettes  which  could  be  broadcast  to  the  schools 
was  also  delayed. 

Participating  Schools 

The  number  of  schools  involved  in  Project  North  grew  during  the  year.  Originally  it 
was  thought  that  there  would  be  13  panicipating  schools.  By  June,  1989  the  number  had 
increased  to  26.  Table  1.1  Hsts  the  participating  jurisdictions  and  the  schools  involved. 
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Table  1.2 

Year  one  schools  and  the  forms  of  technology  implemented  for  instruction. 


School 

CML 

Audicxjonferencing 

80 

88  8Q 

8Q  QO 

Beaverlodge 

1 

1 

La  Crete 

E.  W.  Pratt  (High  Prairie) 

9 

Eaglesham 

Ecole  Routhier 

^| 

Fort  Vermilion  Public 

Fox  Creek 

9 

Glenmary  (Peace  River)  ^ 

Grande  Prairie  Comp 

V 

V 

D 

Grimshaw  Jr-Sr  High  Sch  4 

High  Level 

Hines  Creek 

Holy  Family  (Grimshaw)  ^ 

Kinuso 

V 

Paul  Rowe  (Manning)  ^ 

V 

Rainbow  Lake 

V 

Ridgevalley 

V 

Rocky  Lane 

A 

V 

V 

Roland  Michener  (Slave  L.) 

Savanna 

V 

Sexsmith  Secondary 

St.  Joseph's  (Grande  Prairie) 

a/ 

St.  Thomas  More  (Fairview) 

V 

3 

3 

Upper  Hay  River 

Worsley 

V 

V 

1 .  Plans  under  way  to  take  CML  feed  from  Sexsmith  in  winter  term. 

2.  Using  CML  materials  and  printouts  of  CML  tests  supplied  from  another  school 

3 .  Supplier  but  not  consumer  of  teleconferencing  courses. 

4.  Audiographic  systems  installed  for  future  use. 

5 .  Equipment  present  but  not  used  for  instructional  purposes. 
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If  one  examines  this  list  one  finds  that  it  includes  schools  of  varying  enrollments.  Generally 
the  schools  have  fewer  than  100  high  school  students,  though  there  are  notable  exceptions  such 
as  St.  Joseph's  in  Grande  Prairie  and  Sexsmith  Secondary,  and,  of  course,  Grande  Prairie 
Composite  with  approximately  1400  students.  In  the  case  of  Grande  Prairie  Composite,  the 
interest  in  CML  mathematics  lay  not  as  an  instructional  approach  for  the  entire  school  but  rather 
for  a  very  smaU  subset  of  its  students.  One  also  should  note  two  schools  which  were  part  of 
the  Project  but  in  which  no  distance  education  took  place  during  the  1988-89  school  year,  La 
Crete  and  Spirit  River.  Table  1.2  lists  the  schools  visited  by  the  evaluators  and  indicates  which 
technologies  were  used  in  these  schools  in  each  school  year.  Each  school  was  contacted  by 
telephone  during  November,  1989,  to  ascertain  what  changes  had  been  implemented  after  the 
first  year  of  the  project.  In  some  cases  technology  which  was  tried  during  the  1988-89  school 
year  was  not  continued  into  1989-90,  and,  conversely,  some  schools  implemented  new 
technology  during  1989-90.  Where  teleconferencing  is  indicated  in  Table  1 .2,  the  reader  may 
assume  that  FAX  machines  were  employed  to  transmit  student  assignments  to  an  off-site 
teacher,  either  in  another  school  or  at  the  Alberta  Correspondence  School.  (The  use  of  the 
FAX  machine  in  Project  North  posed  no  problems  for  the  schools  and,  therefore,  little  attention 
will  be  paid  the  use  of  that  technology  in  this  report.  This  contrasts  with  the  evaluation  of  the 
DLSS  Project^  where  the  high  volume  of  pages  being  transmitted  by  FAX  did  pose  some 
problems.) 

Role  of  the  Evaluators 

The  role  of  the  evaluators  in  Phase  1  of  Project  North  needs  some  explanation.  Because 
of  the  circumstances  which  surround  the  beginning  of  any  large  scale  project,  there  was 
concern  that  the  presence  of  external  evaluators  during  the  time  when  products  and  processes 
were  just  being  put  in  place  would  add  an  unnecessary  complication.  On  the  other  hand,  there 
was  the  perception  that  independent  evaluators  might  contribute  information  that  would  be 
useful  to  the  project.  Also,  there  was  a  desire  to  capture  basehne  data  which  might  be  used  in  a 
subsequent  evaluation  of  the  project.  Consequently  the  role  of  the  evaluators  during  Phase  1 
was  to  be  largely  one  of  famiUarizing  themselves  with  the  project,  gathering  baseline  data,  and 
testing  evaluation  methodology.  They  were  also  expected  to  feed  back  to  the  project  leadership 
and  the  Planning  &  Pohcy  Secretariat  information  which  might  be  useful  in  guiding  the  project. 
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The  statement  of  work  for  the  evaluation  listed  four  tasks  : 

1 .  To  conduct  an  introductory  tour  of  the  sites  participating  in  Distance  Learning 
Project  North. 

2.  To  assist  in  the  collection  of  baseline  data  for  1988-89  and  to  develop  interview 
protocols. 

3 .  To  visit  each  of  the  project  sites,  and  meet  with  each  of  the  stakeholders:  project 
management,  superintendents,  principals,  those  responsible  for  coordinating  the 
project,  participating  teachers,  students,  representatives  from  the  cooperating 
agencies,  and  those  responsible  for  developing  the  materials  that  were  used. 

4.  To  produce  a  Phase  1  evaluation  report  which  would  include  proposed  revisions  to 
the  evaluation  questions,  and  a  straight-line  projection  of  how  the  performance  in 
Phase  1  might  be  perceived  within  the  context  of  a  Phase  2  evaluation. 

Because  of  the  need  to  balance  the  provision  of  useful  evaluative  information  with  sensitivity  to 
the  circumstances  surrounding  the  start-up  phase  of  an  ambitious  project,  the  report  which 
follows  will  integrate  the  information  gathered  across  the  entire  project.  Perceived  strengths 
and  weaknesses  of  individual  sites  will  not  be  identified  by  location.  The  reader  should  keep  in 
mind,  however,  that  the  evaluators  found  substantial  commonality  among  the  experiences  of 
the  participating  schools,  and  the  findings  reported  here  ought  not  to  be  attributed  to  a  small 
number  of  exceptions. 


For  additional  information  on  the  DLSS  Project  see,  Clark,  W.B.  and  Schieman,  E.,  (1989)  Evaluation  of 
Phase  1  of  the  Distance  Learning  in  Small  Schools  Action  Research  Project,  Alberta 
Education;  and  also,  Clark,  W.B.  and  Schieman,  E.  (1990)  Evaluation  of  Phase  2  of  the  Distance 
Learning  in  Small  Schools  Action  Research  Project,  Alberta  Education. 
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2.  EVALUATION  METHODOLOGY 


General  Approach 

In  order  to  obtain  an  accurate  understanding  of  the  implementation  of  the  project,  a 
research  design  which  stressed  the  project's  exploratory  and  developmental  nature  was 
undertaken.  A  case  study  approach  was  used  to  ensure  the  recognition  of  contextual  factors 
which  affected  the  project.  This  evolutionary  design  allowed  for  the  inclusion  of  items  which 
the  participants  considered  important  but  which  might  not  have  been  considered  in  the  original 
design  of  the  evaluation. 

Data  Gathering  Techniques 

The  specific  methodologies  employed  were  interviews  with  personnel  involved  in 
various  aspects  of  the  project,  observation  of  project  implementation  in  the  schools,  and 
document  analysis.  The  sites  were  divided  between  the  two  investigators,  and  over  the  course 
of  the  year  two  site  visits  were  made  to  each  participating  school.  During  the  first  visit  the 
principal  and  participating  teachers  were  interviewed,  and  some  in-class  observations  took 
place.  For  the  second  site  visit,  each  investigator  was  accompanied  by  two  research  assistants 
who  interviewed  students  while  the  investigator  conducted  follow-up  interviews  with 
teacher(s)  and  the  principal.  Confirmatory  site  observations  were  also  conducted.  All  but  two 
superintendents  of  the  participating  jurisdictions  involved  in  the  project  were  interviewed  on  at 
least  one  occasion,  as  were  some  central  office  staff.  The  investigators  also  attended  one 
steering  committee  meeting  as  observers  and  spent  time  in  intensive  interviews  with  the  project 
leaders,  other  Alberta  Correspondence  School  staff,  and  representatives  of  Digital  Equipment 
Corporation,  Computer  Based  Training  Systems,  and  ACCESS.  Since  the  site  visits  to  the 
schools  all  took  place  during  the  1988-89  school  year,  telephone  interviews  with  the  principals 
were  conducted  during  November,  1989,  chiefly  to  ascertain  differences  between  the 


implementation  of  distance  education  in  the  schools  in  1989-90  compared  to  the  previous  year. 
In  all,  37  person-days  were  devoted  to  field  work  and  data  gathering. 

Interviews.  The  interview  formats  varied  from  a  series  of  pre- selected  questions  for  the 
student  interviews  (all  instrumentation  in  Appendix  A)  to  semi-structured  interviews  with 
school  staffs  and  project  leaders.  Because  of  the  large  numbers  of  students  who  were  involved 
in  some  schools,  it  was  not  possible  to  interview  each  student.  In  these  cases  a  subset  of 
students  selected  by  the  principal  were  interviewed  individually,  while  the  remaining  students 
were  interviewed  in  focus  groups  of  seven  to  10  students.  Principals  were  requested  to 
identify  for  individual  interviews  those  students  who  represented  a  cross- section  of  the 
population.  As  a  data  gathering  strategy,  focus  groups  were  of  Umited  use.  While  they  yielded 
useful  information  when  suggestions  for  change  and/or  improvement  were  elicited,  they 
proved  less  than  satisfactory  for  gathering  data  on  individual,  personal  experiences.  Student 
data  were  gathered  on  response  sheets  which  the  interviewers  filled  out  during  the  interview. 

In  order  to  ensure  consistency  across  the  project,  the  investigators  shared  their 
experiences  in  using  the  student  interview  guides  and  response  sheets  after  data  had  been 
gathered  at  three  schools.  Based  on  these  discussions,  they  amended  the  interview  schedules 
to  reflect  new  issues  which  had  been  identified  by  students.  For  example,  in  developing  the 
student  interview  protocol  the  evaluators  had  not  anticipated  that  students  would  discover  a 
variety  of  techniques  for  "short-circuiting"  the  way  the  CML  system  was  intended  to  be  used. 
When  they  met,  the  evaluators  compared  responses  from  individual  and  group  interviews  for 
richness  of  data  and  decided  to  continue  with  individual  interviews  wherever  possible.  The 
meetings  of  the  evaluators  and  research  assistants  provided  an  opportunity  to  enhance  inter- 
rater  reliability  of  the  interview  data  through  cross-checking  the  categorization  of  responses  to 
individual  questions. 

The  semi-structured  interviews  with  principals  and  teachers  were  based  on  common 
sets  of  questions  drawn  up  by  the  investigators  but  were  amended  during  the  course  of  the 
conversations  to  follow  up  on  topics  of  interest  to  the  interviewee.  The  sequence  of  site  visits 
allowed  for  the  development  of  questions  based  on  data  collected  as  the  study  progressed. 
These  further  questions  became  the  focus  of  discussion  during  the  second  site  visits.  The  third 
set  of  interviews,  which  were  conducted  by  telephone,  were  based  on  a  pre-arranged  set  of 
questions  that  had  been  sent  previously  by  FAX. 
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Observations.  Time  was  taken  at  each  site  to  visit  the  classroom  used  for  CML  mathematics 
and  to  note  the  physical  arrangements,  the  strategies  for  student  groupings,  the  placement  of 
the  computers,  and  the  charts,  notices,  binders  and  booklets  connected  with  the  mathematics 
program. 

Document  analysis.  Materials  prepared  for  the  various  advisory  groups  and  committees 
were  reviewed  to  ensure  accuracy  of  contextual  information.  Booklets  and  computer-generated 
tests  were  examined  for  confirmation  of  items  arising  from  the  interviews. 

Data  Analysis 

Student  data  were  gathered  on  response  sheets  which  the  interviewers  filled  out  during 
the  interviews.  These  data  were  subsequently  collated  by  school  and  then  reviewed  to  obtain 
patterns  of  student  responses.  In  order  to  ensure  consistency  across  the  project,  the 
investigators  maintained  contact  throughout  the  period  while  interviews  were  being  conducted, 
shared  their  experiences,  and  amended  the  interview  schedules  to  reflect  the  issues  which 
arose.  This  also  provided  an  opportunity  to  enhance  the  reliability  of  the  interviewers. 
Interview  data  were  examined  after  each  round  of  visits  to  identify  categories  of  responses, 
general  themes,  and  issues  or  problems  which  concerned  the  participants.  The  team  members 
met  on  each  of  these  occasions  to  share  their  individual  interview  notes  and  reflections  and  to 
seek  for  consistency  and  clarity  in  the  data  analysis.  At  this  time,  other  topics  or  questions  were 
identified  for  possible  examination.  Following  the  second  round  of  visits,  a  rough 
categorization  of  responses  was  attempted,  and  this  was  retumed  to  participating  principals. 
This  had  two  functions:  (1)  It  provided  a  means  of  assessing  the  accuracy  and  completeness  of 
the  categorized  data,  and  (2)  because  it  displayed  the  variety  of  responses  rather  than 
discussing  the  most  common  items,  it  was  a  potential  learning  device  for  those  principals  who 
wanted  to  know  of  similar  concerns  in  other  contexts  and  how  colleagues  had  solved  them. 

Credibility  of  Data.  Data  credibility  and  dependability  were  maintained  through  cross- 
checks with  knowledgeable  participants  and  through  the  use  of  a  variety  of  information 
sources,  e.g.,  observation  notes,  student  questionnaire  data,  distance  education  instructional 
materials,  to  produce  triangulation  of  the  information.  Further,  preliminary  transcripts  of  the 
analysis  were  circulated  to  principals  for  their  verification. 
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Summary 


An  exploratory  and  iterative  case  study  design  was  used  to  obtain  data  through  observations, 
interviews,  and  document  analysis.  Data  from  the  interviews  were  categorized  and  collated 
with  document  and  observational  information  to  identify  achievements  and  concems  which 
arose  during  the  implementation  and  to  describe  the  processes  used  to  solve  these  concems. 
Triangulation  procedures  and  verification  of  categories  and  themes  through  checks  with  school 
-based  participants  and  other  knowledgeable  personnel  ensured  the  credibility  and 
transferability  of  the  data. 
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3.    DESCRIPTION  OF  FINDINGS 


The  specific  applications  which  formed  the  focus  of  this  study  were  the  implementation 
of  learner-centred  high  school  mathematics  correspondence  school  materials  supported  by  a 
computer  managed  learning  system,  and  the  introduction  of  a  variety  of  other  technologies  to 
enhance  distance  learning  opportunities  generally.  These  technologies  included  audio- 
conferencing,  audiographics,  and  FAX  machines.  Originally,  the  plans  for  implementation 
included  the  provision  of  a  grade  10  French  course  which  would  pilot  the  use  of  interactive 
satellite  broadcasts  with  audioconferencing  as  well  as  prepared  print  materials.  A  continuing 
problem  for  all  the  courses  was  the  limited  amount  of  time  for  course  development.  This  was 
compounded  in  the  French  course  by  the  fact  that  the  provincial  curriculum  was  being  changed 
and  the  new  Program  of  Studies  had  not  been  completed.  Hence,  the  French  10  pilot  was  not 
available  for  implementation. 

Lack  of  sufficient  development  time  was  also  a  factor  in  the  completeness  of  the  CML 
test  bank.  Although  the  test  bank  contained  questions  for  most  of  the  domains  addressed  in  the 
program,  the  number  of  questions  was  very  limited  in  the  early  stages  and  there  were  many 
inconsistencies  in  the  format  required  for  the  student  responses.  Throughout  the  project,  both 
the  CML  test  bank  and  some  of  the  mathematics  materials  were  revised  to  overcome  those 
pedagogical  concerns  which  were  identified  by  Alberta  Correspondence  School  staff  and 
participating  teachers. 

While  FAX  machines  and  the  MicroVAX  computers  and  terminals  were  in  place  in  the 
participating  schools  in  September,  1988,  the  audioconferencing  equipment  did  not  arrive  until 
February,  1989.  Hence,  while  almost  all  schools  participated  in  the  CML  mathematics 
implementation  and  used  FAX  machines  to  speed  the  tumaround  time  for  students  taking 
correspondence  school  courses,  only  a  few  schools  piloted  the  Social  Studies  13  course,  or 
used  audioconferencing.  The  audiographics  capabiHty  was  not  used  mainly  due  to  technical 
problems  in  set  up  at  the  sites  involved.  (There  are  plans  to  use  the  audiographics  system  to 
support  the  teaching  of  Mathematics  31  between  St.  Thomas  More  High  School  in  Fairview 
and  Glenmary  High  School  in  Peace  River  in  the  second  term  of  1989-90.) 


CML  Mathematics 

The  CML  mathematics  implementation  raised  a  number  of  issues  for  teachers,  students, 
and  principals  at  the  school  level  These  have  been  categorized  under  the  following  headings. 
Because  of  the  iterative  nature  of  the  project,  many  of  the  concerns  raised  by  individuals  in 
June,  1989  have  been  ab-eady  addressed  in  this  school  year  (1990-91). 

The  Booklets.  Teachers  who  had  reviewed  the  booklets  commented  favorably  on  their 
comprehensiveness  in  covering  all  topics.  They  recognized  that  all  students  would  be  required 
to  cover  all  topics  thereby  putting  them  on  a  par  with  students  throughout  the  province.  Some 
commented  that  many  assumptions  about  students'  capabilities  had  not  been  spelled  out  in  the 
print  materials  as  clearly  as  they  wished,  and  that  such  assumptions  sometimes  did  not  reflect 
the  reality  of  their  students'  mathematical  competencies. 

An  examination  of  the  booklets  showed  some  general  weaknesses,  however,  in  their 
instructional  design.  Teachers  and  students  identified  these  weaknesses  when  they  requested 
more  examples,  greater  synthesis  of  content  across  modules,  and  more  attention  to  student 
understanding  of  mathematical  processes.  Teachers  also  expressed  concerns  about  student 
motivation  which  were  linked  to  the  ways  students  used  the  booklets.  Many  booklets  followed 
a  standard  format  in  which  the  general  content  was  presented,  followed  by  a  series  of  examples 
to  be  worked  on  by  the  student.  This  format  did  not  match  the  classroom  pedagogy  used  in 
teaching  mathematics  to  which  the  students  were  accustomed.  In  the  familiar  format,  teachers 
worked  through  examples  from  the  particular  to  the  general,  thus  providing  a  framework  for 
the  development  and  confirmation  of  principles  and  procedures.  Only  then  were  students  given 
appUcation  items.  This  departure  from  the  familiar  had  two  impacts:  (1)  the  students  tended  to 
skip  the  introduction  of  content  and  begin  with  the  examples,  and  (2)  teachers  tried  to  provide 
oral  overviews  to  units  and  booklets  to  help  students  focus  on  the  important  aspects.  Taken  to 
its  extreme,  some  teachers  used  the  materials  as  a  resource  only  and  taught  their  classes 
directly,  reserving  the  examples  and  tests  for  individual  seat  work. 

As  the  booklets  have  been  revised  by  ACS,  a  number  of  changes  have  been  made:  the 
pedagogy  has  been  re-examined  and  the  general  design  and  visual  format  have  been  improved. 
However,  there  are  still  fewer  examples  to  choose  from  than  some  teachers  desire.  The  issue  of 
the  quality  of  the  pedagogical  design  of  the  materials  remains  a  crucial  factor,  especially  where 
teachers  are  not  mathematics  specialists  and  may  depend  on  the  booklets  to  carry  the  content. 

Across  the  project,  teachers  used  a  variety  of  approaches  to  managing  CML 
mathematics.  In  some  schools,  students  were  required  to  complete  the  booklet  before  being 
allowed  to  access  the  test,  while  in  others  they  did  not  use  the  booklet  except  as  a  resource  after 
they  had  completed  an  initial  test.  In  units  where  the  relationship  between  test  bank  items  and 
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the  booklet  content  was  strained,  some  teachers  gave  up  requiring  students  to  work  in  the 
booklets  since  the  mismatch  between  the  practice  and  test  items  led  more  to  confusion  than 
learning.  In  contrast,  other  teachers  were  adamant  that  working  through  the  booklets  gave  the 
student  a  much  more  thorough  grounding  in  the  principles  of  mathematics.  The  teacher's 
choice  of  approach  may  also  have  been  influenced  by  some  of  the  following  factors. 

Computerized  testing.  The  paucity  of  items  in  the  test  bank  gave  rise  to  a  number  of 
problems  which  will  be  easily  alleviated  as  the  bank  is  expanded.  In  some  areas,  the  items  in 
the  bank  were  too  difficult  for  the  grade  level  or  did  not  correspond  to  the  level  of  difficulty  of 
the  examples  in  the  booklet.  There  were  few  items  at  the  Mathematics  30  level  which  used 
wording  similar  to  that  of  the  Diploma  examination.  Some  domains  were  tested  inadequately 
and  problem- solving  items  were  noticeably  few. 

These  weaknesses,  in  turn,  gave  rise  to  compensatory  activities  by  teachers,  most 
frequently  through  the  provision  of  additional  questions.  Students,  recognizing  the  likelihood 
that  consecutive  tests  would  contain  similar  and  sometimes  the  same  questions,  and  that  test 
scores  were  the  major  indicator  of  achievement,  chose  to  use  the  tests  as  organizers  for  their 
leaming.  Where  allowed,  they  drew  a  test,  worked  on  the  incorrect  responses  (sometimes 
using  the  booklets  as  a  resource)  and  then  drew  a  second  test.  Because  computer  testing  was 
seen  as  analogous  to  the  elimination  of  teacher  marking,  teachers  were  unprepared  for  the 
number  and  variety  of  questions  which  they  had  to  mark  themselves.  These  varied  from 
booklet  practice  questions  where  the  student  could  not  find  the  error  to  test  bank  items  where 
process  was  the  focus.  Perhaps  it  was  the  variety  of  questions  which  teachers  had  to  handle 
within  a  class  period  which  contributed  to  the  teacher's  frustrations  with  marking,  since  it 
meant  that  they  had  to  be  able  to  work  in  any  area  of  the  curriculum  and  at  any  grade  level  at  a 
moment's  notice.  Where  teachers  were  not  mathematics  speciaHsts,  this  was  an  especially 
difficult  task.  Because  students  needed  the  feedback  in  order  to  know  whether  they  could 
proceed  to  the  next  unit,  some  students  refused  to  move  ahead  until  their  work  was  marked  and 
incidentally  gave  themselves  some  spare  time.  In  such  circumstances,  however,  teachers 
tended  to  over-ride  the  computer's  control  so  that  students  could  continue.  Some  teachers 
commented  that  the  provision  of  a  pre-test  would  allow  students  to  set  up  a  challenge  for 
themselves  and  enhance  their  motivation  to  work  through  the  booklets. 

Another  suggestion  was  to  increase  the  use  of  the  messaging  function  to  provide  a 
motivational  bonus  for  students.  Students  also  proposed  the  addition  of  Mathematics  30 
examples,  the  inclusion  of  Diploma  practice  tests,  the  use  of  a  review  test  if  a  student  failed  a 
supervised  test,  and  the  inclusion  of  processes  as  well  as  answers  in  feedback  on  tests.  The 
programming  capability  to  address  most  of  these  issues  was  already  present  but  perhaps 
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teachers  were  not  aware  of  the  suggestions  or  decided  not  to  implement  them  in  this  first  trial 
year. 

The  Mathematics  30  diploma  exams  are  a  somewhat  sensitive  subject  among  schools. 
Most  schools  reported  that  students  who  wrote  Mathematics  30  Diploma  examinations  in  June, 
1989  did  not  do  as  well  as  expected.  Most  schools  also  indicated  that  there  may  be  alternate 
explanations  for  the  lower  marks  than  the  fact  that  the  students  took  CML  mathematics.  On  the 
other  hand,  the  fact  that  this  experience  was  common  across  so  many  of  the  schools  suggests 
that  the  preparation  for  the  Diploma  examinations  is  a  matter  warranting  close  attention. 

Computer  Equipment.  After  the  initial  start-up  in  most  schools,  there  appeared  to  be  few 
problems  with  the  hardware  or  software.  Where  there  were  difficulties,  the  Distance  Learning 
Implementation  Coordinator  (Rik  Hall)  provided  a  fast,  effective  service.  Some  schools  had 
three  terminals  hooked  to  their  MicroVAX  computer,  and  when  all  were  in  operation,  the  speed 
of  operations  slowed  down  substantially.  This,  in  turn,  created  problems  of  access  for 
students  in  the  classroom.  In  general,  having  only  two  terminals  on  the  MicroVAX  was 
necessary  if  15-18  students  were  to  have  ready  access  to  the  test  bank.  Digital  Equipment 
Corporation  indicates  that  new  software  which  was  loaded  on  the  2000  series  computers  now 
runs  more  quickly  than  what  was  available  in  1989.  Also,  the  3000  series  which  are  now 
available,  are  supposed  to  be  much  faster  machines. 

Computer  management.  Because  of  the  limited  time  available  for  the  construction  of  the 
test  bank  items,  previously  developed  test  banks  were  used  in  addition  to  newly  constructed 
items.  Unfortunately,  there  was  a  lack  of  consistency  in  the  format  in  which  students  were 
required  to  enter  their  answers.  For  example,  sometimes  the  answer  was  required  to  the  fourth 
decimal;  other  items  required  that  the  answer  be  rounded  off.  Where  students  used  an 
alternative  way  of  stating  the  answer,  (not  reducing  fractions  was  one  of  the  most  common), 
this  answer  was  not  accepted  as  correct.  Hence,  in  the  fall  term,  there  was  much  frustration 
and  teacher  time  expended  on  identifying  inaccurate  responses,  correcting  the  "errors"  on 
individual  student  examinations  and  reclaiming  student  grades.  The  amount  of  time  teachers 
put  into  these  activities,  and  their  concern  that  their  students  not  be  penalized,  cannot  be 
overestimated.  Teachers  spent  time,  both  in  class  and  as  much  as  one  hour  after  school  every 
day,  changing  grades  and  learning  to  operate  the  programs  and  to  access  student  history  data. 
A  few  teachers  worked  their  way  through  all  of  the  tests  at  each  grade  level  and  provided  a 
binder  which  students  could  consult  to  see  the  format  required. 

Many  of  these  problems  have  now  been  eliminated,  and  most  teachers  feel  reasonably 
"competent"  with  the  system;  however,  the  diagnostic  functions  of  the  computer  program  are 
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generally  under-utilized  and  teachers  are  only  beginning  to  understand  how  they  can  be 
integrated  into  their  program.  Few  teachers  use  the  program  to  identify  areas  of  student 
weakness  so  that  they  can  provide  small  group  remediadon.  This  is  one  area  where  teachers 
can  use  the  computer  to  do  the  initial  selection  rather  than  depending  on  their  own  memory  of 
student  errors.  A  manual  which  would  outiine  these  possibilities  and  show  the  programming 
necessary  to  access  these  functions  is  now  in  preparation. 

Scheduling.  With  respect  to  scheduling,  schools  varied  along  three  dimensions.  (1)  They 
varied  in  the  extent  to  which  they  formally  scheduled  mathematics  periods,  ranging  from 
schools  with  no  designated  periods,  to  those  with  some  assigned  periods,  to  those  with  all 
assigned  classes.  (2)  They  also  varied  in  the  composition  of  these  classes  where  they  ranged 
from  classes  of  mathematics  students  from  all  grades  to  those  with  specific  combinations  of 
grades,  to  those  designated  by  grade  level,  e.g.,  lO's.  (3)  Class  size  was  the  third  variable 
with  groups  ranging  from  a  high  of  34  to  those  with  5  students. 

These  three  factors  combined  to  form  variations  not  all  of  which  were  equally 
successful.  Students  and  school  staff  generally  preferred  at  least  some  scheduled  times.  They 
also  found  it  best  to  limit  classes,  thereby  reducing  the  range  of  possible  tutor  topics  for  the 
teacher.  Some  did  this  by  confining  the  class  to  one  grade  level  while  others  combined  no 
more  than  three  different  learning  combinations  (e.g.,  31,  23,  10).  The  larger  the  class  size  at 
one  grade  level,  the  more  likely  it  was  that  the  teacher  had  to  group  the  students  rather  than 
work  with  them  individually  to  provide  assistance.  The  number  of  students  doing  each 
variation  of  mathematics  at  that  grade  level  was  also  a  factor.    There  was  some  indication  in 
Phase  1,  incidentally,  that  a  maximum  of  approximately  20  students  (some  would  say  16) 
seems  to  be  an  appropriate  number  until  the  procedures  for  assisting  students  become  more 
streamlined  and  the  speed  of  computer  test  generation  quickened.  After  their  experiences  in 
1988-89,  most  schools  are  much  more  aware  of  the  potential  problems  raised  by  each 
combination  and  have  taken  steps  to  ensure  a  reasonable  class  in  size  and  grade  diversity. 

While  the  fluidity  in  enrolment  of  grades  1 1  and  12  continues  to  be  an  uncontrollable 
factor  for  small  rural  high  schools,  principals  now  are  able  to  offer  students  the  opportunity  to 
obtain  credits  regardless  of  timetable  or  employment  conflicts.  Many  principals  spoke  of 
individual  arrangements  which  had  been  highly  beneficial  to  the  students  and  which  the  school 
could  not  have  accommodated  in  any  other  way. 

Learning  management.  The  specific  learning  management  structures  which  teachers 
implemented  reflected  most  strongly  the  philosophical  orientation  of  the  introductory  inservice. 
These  teachers  had  heard  that  they  would  be  facihtators  of  learning  who  provided  individual 
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tutoring  to  students  learning  at  their  own  rates.  (Some  believed  that  the  CML  system  was 
actually  a  computer  assisted  learning  or  CAL  system.)  They  began  the  year  by  spending  time 
discussing  student  responsibilities  including  the  sequence  of  the  course  content,  the  number  of 
modules,  the  use  of  the  computer  as  a  testing  tool  and  the  importance  of  working  consistently 
to  finish  the  course  in  the  time  provided.  Many  developed  a  time  line  based  on  completion  dates 
for  the  units  at  that  grade  level  and  made  posters  identifying  these  dates  so  that  students  could 
easily  calculate  the  difference  between  where  they  were  and  where  they  were  supposed  to  be. 
Because  the  provision  for  individual  student  progression  which  was  emphasized  as  a  core 
value  of  the  program  was  one  most  teachers  espoused,  they  did  not  intervene  and  enforce 
deadlines  by  which  students  had  to  have  completed  each  unit.  The  notion  of  "going  at  one's 
own  rate"  was  taken  to  apply  to  classroom  time.  In  fact,  many  students  did  no  homework  and 
did  not  use  their  spares  to  meet  the  unit  dates  necessary  for  course  completion.  While  some 
teachers  focused  on  keeping  students  on  task  and  introduced  the  topics  of  the  booklets  in  mini- 
lessons,  others  were  besieged  by  forests  of  hands  and  spent  their  entire  periods  answering 
questions  and  marking  or  remarking  tests  and  examinations. 

Of  importance  was  whether  teachers  demanded  that  booklets  be  completed  prior  to 
drawing  a  test.  Some  teachers  just  advised  students  to  do  the  booklet  while  others  gave  a  grade 
for  the  completed  booklet;  some  allowed  students  to  pull  a  test  but  wanted  to  see  the  completed 
work  in  the  booklet  before  assisting  students  with  their  mistakes.  Other  teachers  started  by 
requesting  completion  of  booklets  and  then  found  themselves  and  their  students  frustrated  by 
the  lack  of  congruity  between  the  booklets  and  the  tests  for  some  units  and  consequently  gave 
up  the  procedure.  Where  teachers  were  working  with  20  or  more  students,  supervision  of 
booklet  completion  was  onerous  and  time  consuming.  In  many  instances,  students  were 
unwilling  to  proceed  unless  the  teacher  had  marked  the  booklet  or  the  test  exercises.  This  led 
to  unstructured  time  for  the  student  and  a  potential  source  of  disruptive  behavior  in  the 
classroom. 

Unwilling  to  disrupt  the  option  of  "own  rate"  progression,  some  teachers  used  grading 
schemes  to  structure  students'  work.  These  schemes  included  allocating  marks  for  completing 
the  booklets  or  counting  the  mark  received  on  the  first  of  the  practice  exercises  toward  the 
course  grade  but  still  requiring  80%  in  three  tries  for  competency.  Other  teachers  assigned  a 
percentage  of  the  final  mark  to  each  test.  All  these  procedures  were  designed  to  discourage 
students  from  drawing  tests  before  attempting  the  booklets.  Where  students  were  required  to 
complete  the  booklets,  there  were  fewer  complaints  about  the  lack  of  test/booklet  correlation. 
In  some  instances,  teachers  pulled  review  tests  prior  to  the  teacher-supervised  test  to  help 
students  review  several  booklets  at  once. 
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Teacher  assistance.   Where  teachers  had  classes  of  26  or  more  students,  or  where  all 
students  who  had  spares  had  the  option  of  attending  the  mathematics  room,  teachers  had 
difficulty  meeting  all  the  students'  requests  for  assistance.  One  school  had  provided  and  others 
planned  to  provide,  a  student  with  good  peer  tutoring  skills  who  had  completed  Mathematics 
30  and  could  earn  work  experience  credits  by  providing  student  assistance.  In  another  school 
where  the  Micro  VAX  was  heavily  utilized,  an  adult  was  employed  to  enter  student  test 
responses  into  the  CML  system  and  thereby  reduce  the  lag  time  while  students  waited  for 
access  to  the  computer.  While  it  solved  the  "wait"  problem,  it  also  denied  the  student  access  to 
the  computer  and  the  immediacy  of  response  which  was  a  positive  motivational  feature  for 
some  students.  In  one  school,  the  adult  aide,  at  the  request  of  the  teacher,  used  the  computer  to 
obtain  diagnostic  information  from  the  test  results.  From  this  information,  the  teacher  then 
identified  the  small  group  of  learners  who  needed  further  work.  Tasks  for  assistants  varied 
from  non-instructional  activities  such  as  entering  tests  and  pulling  diagnostic  information  and 
student  histories  to  situations  where  the  focus  was  on  individual  student  assistance  and  where 
the  aide  identified  for  the  teacher  those  students  with  a  common  concern. 

Classroom  environment.  Not  all  teachers  seemed  equally  aware  of  the  potential  impact  of 
the  classroom's  physical  environment.  The  suggestions  given  to  teachers  at  the  preliminary 
workshop  provided  some  options,  but  most  often  the  location  of  computer  plugs  and  telephone 
jacks  were  significant  factors  in  room  organization.  Some  teachers  clustered  desks  or  tables  in 
an  oval  or  horseshoe  format  with  the  teacher's  desk  at  the  open  end  while  others  had  tables  or 
desk  clusters  placed  throughout  the  room.  Some  formats  made  teacher  supervision  more 
difficult.  Having  a  space  set  aside  for  the  teacher  supervised  tests  was  also  important  in 
helping  the  teacher  control  the  situation  in  the  classroom.  This  is  another  area  where  evident 
changes  have  been  made  this  year,  following  the  experiences  in  1988-89.  Where  the  social 
organization  of  the  class  is  not  controlled  by  direct  focus  on  the  teacher's  voice  and  actions, 
then  the  physical  structure  of  the  classroom  is  important  in  conveying  the  message  of  a 
common  focus  on  task  completion. 

Cooperative  learning.   In  many  schools,  students  developed  informal  learning  groups 
whose  members  provided  support  and  assistance  to  each  other.  Many  teachers  commented 
positively  on  the  increase  in  peer  tutoring  and  in  task-focused  conversation.  They  noted  that 
where  students  formed  supportive  work  groups,  they  were  more  likely  to  keep  to  the  schedule 
and  because  they  all  worked  on  the  same  booklets,  it  was  easier  for  the  teacher  to  provide  a 
mini-lesson  to  introduce  the  concept  and  ensure  that  it  was  clearly  understood.  A  small  number 
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of  teachers  indicated  that  the  conversation  in  such  groups  tended  to  be  more  social  than 
academic. 

Tutoring-Teaching.  From  the  initial  inservice,  teachers  clearly  expected  that  their  time 
would  be  taken  up  in  tutoring  individual  students  rather  than  in  didactic  teaching.  One  teacher 
commented  after  this  introduction,  "This  is  management,  where  is  the  teaching?"  Despite  a 
desire  to  go  over  the  concepts  and  explain  them  in  order  to  ensure  that  students  fully 
understood  the  topics,  teachers  in  general  avoided  full  class  lectures. 

The  transfer  of  students  from  a  system  where  the  teacher  provided  the  impetus  through 
a  class  lecture,  and  sustained  the  focus  through  seat  work,  to  one  where  students  had  to  obtain 
all  the  information  from  written  booklets  and  provide  their  own  motivation  with  limited  teacher 
assistance  was  a  difficult  transition  for  many  students.  Some  teachers  felt  that  introductory 
sessions  were  essential  to  ensure  students  had  the  necessary  background  and  motivation  but 
found  these  difficult  to  continue  as  students  spread  out  through  the  units. 

There  were  difficulties  at  the  beginning  of  the  school  year  in  both  1988-89  and  in 
1989-90.  In  September,  1988,  this  was  due  to  the  late  arrival  of  the  print  materials  which 
came  in  mid  October.  There  was  a  similar  delay  in  Fall,  1989  because,  following  revisions, 
there  was  insufficient  time  to  produce  enough  booklets  and  the  numbers  of  schools  requesting 
materials  escalated.  Fortunately,  the  delay  was  not  as  long  as  in  the  previous  year. 

Most  teachers  confined  themselves  to  working  with  individual  students.  In  organizing 
for  this  school  year,however,  most  planned  to  do  more  direct  teaching.  Some  thought  that  they 
would  introduce  new  concepts  to  the  entire  class  as  the  bulk  of  the  students  began  a  new  unit, 
with  slower  students  taking  notes  which  the  teacher  would  go  over  later,  and  with  faster 
students  using  it  as  a  review.  Some  planned  to  keep  students  in  cohorts  so  that  they  could 
monitor  them  more  closely.  Some  talked  about  dropping  the  three-part  competency  provision 
and  replacing  it  with  a  once-only  test.  While  these  options  would  seem  to  address  a  number  of 
issues — keeping  students  moving  forward  at  a  reasonable  pace  and  ensuring  that  all  students 
had  a  full  understanding  of  the  concepts — they  would  limit  the  full  benefit  of  the  program: 
individual  student  progress.  The  drive  to  include  direct  teaching,  either  to  small  groups  or  to  a 
total  class,  did  not  result  so  much  from  teachers'  desire  to  teach  as  from  a  concern  that  the  print 
materials  did  not  introduce  the  topics  appropriately.  There  was  a  concern  that  weaker  students 
in  particular,  were  learning  to  focus  on  mathematical  procedures  rather  than  principles,  and  that 
students  did  not  see  connections  or  were  unable  to  address  problems  if  they  were  stated  in  a 
format  other  than  the  one  they  had  worked  on  in  the  booklet.  Because  the  Mathematics  30  test 
bank  in  1989-90  did  not  include  questions  in  the  same  format  as  appeared  in  the  Diploma 
examinations,  many  teachers  did  intensive  coaching  with  students  to  ensure  that  they  were 


18 


comfortable  with  the  question  structure  and  felt  prepared  for  the  examination.  If  teachers  were 
convinced  that  students'  needs  were  well  served,  they  might  then  fmd  ways  to  encourage 
students  to  maintain  a  steady  pace  other  than  by  direct  intervention  in  their  learning. 

Student  Motivation.  This  was  a  topic  of  major  interest  for  school  staffs  and  students  alike. 
All  felt  that  students  of  whatever  ability  did  well  using  this  format  if  they  were  self-motivated. 
Their  major  concern  was  for  the  unmotivated  students — especially  those  who  didn't  like  either 
mathematics  or  reading.  Without  a  support  structure  to  guide  their  learning,  many  students 
were  unable  to  cope  with  their  newfound  freedom.  Some  found  the  first  two  units  relatively 
easy,  and  they  relaxed,  believing  that  catching  up  would  not  be  difficult.  Others  gave  up  when 
they  found  that  the  format  of  the  units  was  similar,  and  boredom  set  in.  For  others,  the  amount 
of  reading  required  was  their  undoing.  These  students  were  used  to  an  "oral/  visual"  pedagogy 
and  found  the  transfer  to  reading  to  be  difficult  since  it  involved  another  level  of  abstraction. 

These  poorly  motivated  students  provided  the  biggest  challenge  and  concern  for 
teachers.  They  used  up  teacher  time  because  they  needed  the  longest  explanations  and  yet  were 
readily  distracted  if  the  teacher  was  working  with  another  student.  Some  found  ways  to  "beat 
the  system"  by  pulling  tests,  comparing  answers  and  seeking  help  from  others.  Only  as  a  last 
resort,  did  they  consult  the  booklets.  Asked  to  explain  what  they  did  if  they  did  not  pass  the 
competency  test  the  first  time,  the  majority  of  these  students  immediately  drew  a  second  test.. 
In  comparison,  however,  some  motivated  students  used  the  system  to  accelerate  their  progress 
and  complete  two  or  more  mathematics  courses  in  one  year.  These  students  generally  enjoyed 
being  able  to  pace  themselves. 

Reporting-Monitoring.  Most  teachers  began  by  using  the  grade  guide  in  the  booklets  and 
then  amended  it  when  they  encountered  problems.  Teachers  were  somewhat  frustrated  because 
they  felt  responsible  for  the  academic  achievements  of  their  students  but,  because  students 
controlled  the  pace  of  their  learning,  saw  themselves  as  unable  to  require  activities  which 
would  ensure  at  least  a  minimum  level  of  competency.  "Working  at  one's  own  rate"  meant 
that,  in  general,  teachers  monitored  which  units  students  were  on,  but  only  when  Uttle  or 
nothing  was  accompUshed  over  the  course  of  a  reporting  period,  did  they  approach  the 
principal. 

Teachers  found  the  management  function  of  the  computer  program  very  helpful  in 
keeping  track  of  students'  progress.  While  some  teachers  pulled  the  information  weekly,  most 
collected  histories  about  once  per  month.  Some  shared  these  with  their  students  and 
encouraged  them  to  show  them  to  their  parents,  but  teachers  were  aware  that  this  did  not 
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always  happen.  This  was  one  feature  which  was  used  heavily  at  the  beginning  of  the  school 
year  but  was  gradually  ignored  as  the  amount  of  marking  increased  exponentially. 

Many  teachers  knew  which  units  students  were  on  but  did  not  always  keep  track  of 
how  long  they  had  been  working  on  that  unit.  While  some  teachers  sent  home  to  parents 
regular  histories  reporting  the  unit  the  student  was  on  and  the  marks  for  the  competency  tests  to 
date,  others  also  included  'expected  progress'  information  so  that  parents  knew  what  was 
expected. 

The  issues  of  course  failure  or  continuation  have  been  variously  addressed.  Where 
students  were  a  unit  or  two  short  of  course  completion,  some  teachers  planned  to  give  a  review 
test  in  September  and  then  allow  the  student  to  continue  from  that  competency  level.  Others 
encouraged  selected  students  to  work  on  their  booklets  over  the  summer.  In  fact,  students  who 
did  not  complete  are  continuing  their  courses  either  as  a  regular  member  of  their  scheduled 
class  or  in  their  spare  periods.  Happily,  many  students  who  did  little  work  on  the  course  last 
year  have  already  completed  that  course  and  are  working  to  catch  up  with  their  classmates. 
Students  who  had  only  one  or  two  units  completed,  were  counselled  either  to  work  at  a  lower 
level  or  repeat  the  entire  course.  This  has  not  been  a  difficulty. 

Audioconferencing 

Audioconferencing  was  not  heavily  used  during  Phase  1,  but  a  number  of  schools  did 
employ  it  with  a  small  number  of  students.  Courses  which  were  supported  at  least  in  part  by 
audioconferencing  were  Accounting  10,  Business  Education  20,  Biology  30,  German, 
Mathematics  31,  and  Physics  20. 

Whereas  the  introduction  of  the  mathematics  program  occasioned  teachers  to  reorganise 
the  ways  they  structured  their  teaching,  audioconferencing  called  for  the  development  of 
teaching  strategies  which  were  not  a  regular  part  of  their  teaching  repertoire.  Accustomed  to 
monitoring  classes  through  non-verbal  cues  and  using  the  chalkboard  as  a  focus  for  didactic 
instruction,  teachers  had  to  depend  on  print  materials  to  a  much  greater  extent.  Those  who  did 
not  have  prepared  correspondence  materials  had  to  develop  such  materials.  But,  not  only  had 
teachers  to  obtain  or  develop  materials,  they  also  had  to  learn  how  to  discipline  their  own  on- air 
time  and  pace  the  instruction,  to  vary  their  instructional  strategies  to  include  a  greater  variety  of 
voices  and  activities  than  is  necessary  in  the  classroom,  and  to  monitor  student  learning  and 
engender  discussion  using  verbal  cues  alone. 

Audioconferencing  also  requires  appropriate  hardware  set  up  in  convenient  locations 
conducive  to  learning.  Some  schools  had  not  given  sufficient  thought  to  the  ramifications  of 
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location  and  had  put  the  telephone  hook-up,  the  fax  machine  and  the  CML  terminals  beside 
each  other.  Another  important  consideration  was  the  cost  of  long-distance  telephone  calls. 
Especially  at  the  high  school  level  where  these  schools  are  likely  to  be  geographically  far  apart, 
inter-school  cooperative  teaching  means  regular  long-distance  costs. 

Audioconferencing  was  most  effective  in  providing  tutoring  for  students  who  were 
doing  Alberta  Correspondence  School  courses.  In  these  instances,  the  teacher  called  the 
students  and  dealt  with  any  general  questions  first  before  providing  individual  assistance. 
These  sessions  were  usually  about  30  minutes  long  for  4  or  5  students  and  did  not  entail  the 
need  for  a  supervisory  teacher  at  the  receiving  site.  Individual  students  occasionally  called  the 
teacher  for  help  at  other  times.  This  year,  1989-90,  a  number  of  the  schools  which  tried  audio- 
conferencing  in  Phase  1  are  not  using  it.  However,  schools  are  beginning  to  use  the  equipment 
to  enhance  regular  classroom  instruction,  such  as  "bringing  in  "  guests  to  the  class  by 
telephone,  and  for  informal  student  council  meetings  between  schools.  In  one  school  district. 
Mechanics  12,  Accounting  30,  Physics  10  and  French  10  are  being  offered  by 
audioconferencing.  These  experiences  will  prove  valuable  in  identifying  those  skills  which 
teachers  need  to  address  if  the  system  is  to  be  successful  for  learners  and  teachers.  It  should  be 
noted  that  some  schools  which  employed  audioconferencing  last  year  are  not  doing  so  this 
year.  The  reasons  associated  with  dropping  it  appear  to  be  related  to  the  management  and 
pedagogical  issues  noted  above.  In  the  schools  which  have  discontinued  it,  there  was  also  a 
pattern  in  which  (1)  the  teachers  were  doing  the  audioconferencing  courses  in  addition  to  their 
regular  class  assignments  with  no  extra  remuneration  and  (2)  were  tutoring  the  subject,  but  the 
student  assignments  were  being  given  and  marked  elsewhere. 

Level  of  Satisfaction 

The  introduction  of  CML  mathematics  has  been  generally  welcomed  by  principals  and 
teachers  because  it  provides  for  options  which  would  not  be  possible  in  any  other  way.  Not 
only  does  it  expand  the  number  of  courses  available,  but  more  importandy,  principals  are  now 
able  to  offer  students  courses  more  suited  to  their  abilities.  Formerly,  in  a  school  with  seven  or 
eight  Mathematics  30  students  and  two  more  suited  to  Mathematics  33,  the  teacher  had  either  to 
accommodate  the  materials  to  the  range  of  ability  levels  and  hope  the  students  could  manage  to 
keep  up  or  suggest  that  they  work  on  correspondence  materials.  Most  often  such  students  did 
not  take  mathematics  classes  at  all.  Much  of  the  student  population  in  the  nonhem  Albena  is 
transient,  and  through  the  use  of  correspondence  school  materials,  principals  now  have  a 
means  of  helping  students  complete  courses  for  credit  rather  than  having  them  lose  their 
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semester's  work.  Besides  the  organizational  benefits,  principals  and  teachers  were  also 
pleased  at  the  growth  in  maturity  as  students  took  responsibility  for  their  own  learning  and 
showed  enthusiasm  for  their  work.  Asked  if  they  would  recommend  the  CML  approach  to 
others,  students  talked  about  the  importance  of  motivation  and  being  willing  to  work  with  little 
direct  assistance.  Thirty-six  percent  recommended  it  outright,  and  another  32  percent  agreed, 
with  the  addition  of  the  motivational  caveat. 

Audioconferencing  has  been  used  by  too  few  teachers  and  students  to  have  been  fully 
explored.  Where  it  has  been  used  for  tutoring,  students,  teachers  and  principals  are  pleased 
with  the  opportunity  it  affords  to  provide  assistance  to  students.  Its  use  as  a  technology  to 
provide  regular  classroom  instruction  is  in  the  process  of  being  tested  this  year. 

Summary 

In  general,  the  organization  and  structure  necessary  for  successful  implementation  of 
CML  mathematics  has  been  identified  and  discussed  in  the  categories  identified  above.  Of 
continuing  interest  are  the  pedagogies  which  arise  from  using  the  correspondence  materials 
within  the  regular  classroom.  In  every  case  where  the  instruction  is  based  on  correspondence 
materials,  the  importance  of  the  pedagogical  design  of  the  booklets  and  associated  media  cannot 
be  over-emphasized. 
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4.  RECOMMENDATIONS  &  COSTS 


In  many  ways  the  findings  reported  in  Chapter  3  are  themselves  the  recommendations 
from  this  evaluation.  There  are,  however,  certain  global  areas  upon  which  the  evaluators  wish 
to  comment 

Materials  Design 

First,  the  evaluators  commend  the  Alberta  Correspondence  School  and  all  those  who 
worked  on  developing  and  revising  the  materials  for  their  responsiveness  to  feedback  from  the 
field  The  mathematics  course  materials  in  the  schools  in  1989/90  were  an  extensively  revised 
version  of  those  in  use  in  Year  one  of  the  project  Based  on  Year  one  data,  a  small  number  of 
students  commented  (a)  that  they  found  the  printed  materials  followed  a  predictable  pattem 
which  over  a  number  of  lessons  became  monotonous  and  (b)  that  efforts  at  humour  in  some  of 
the  booklets  occurred  in  places  which,  rather  than  adding  levity,  caused  them  fiaistration.  A 
number  of  teachers  commented  that  they  found  the  materials  strong  for  teaching  procedures  but 
weak  for  helping  students  identify  and  understand  the  underlying  principles  and  recognize  their 
potential  application  in  new  situations.  Teachers  also  commented  that  the  approach  taken  in 
presenting  new  content  differed  substantially  fi'om  that  to  which  students  were  accustomed. 
From  such  perceptions  and  a  general  review  of  the  findings,  the  following  recommendations 
are  drawn: 

1 .  The  continued  up-grading  of  the  print  materials  to  deUneate  learning  objectives, 
cognitive  linkages,  required  background  knowledge  and  competencies,  and  evaluation 
options,  as  well  as  closer  attention  to  a  variety  of  learning  strategies  for  differing  ability 
levels  is  applauded  and  should  be  extended  to  all  distance  learning  courses. 

2 .  Developers  of  instructional  materials  should  be  sensitive  to  the  fact  that  research  has 
shown  that  instructional  materials  which  follow  a  predictable  pattem  tend  to  be 
demotivating,  and  should  vary  the  presentation  form. 


3 .  Humour  should  be  used  judiciously,  be  relevant  to  the  subject  matter,  and  be  placed  at 
points  where  students  can  be  expected  not  to  be  experiencing  frustration. 

4 .  Curricular  goals  which  emphasize  principles  or  the  learning  process  should  be  explicitly 
identified  in  the  printed  materials;  and  content  addressing  them  should  be  labelled  as 
such. 

5 .  The  materials  should  be  examined  to  see  if  the  introduction  of  new  content  indeed 
moves  "from  the  known  to  the  unknown",  "from  the  concrete  to  the  abstract."  Since 
each  content  area  requires  particular  pedagogical  strategies,  these,  too,  must  be  part  of 
the  course  design. 

6 .  Students  should  be  oriented  to  the  organization  of  the  printed  materials  and  taught  how 
to  use  them  effectively. 

In  addition,  the  evaluators  would  make  seven  other  suggestions  pertaining  to  the  materials: 

7 .  Instructional  materials  should  be  prepared  which  assist  students  to  synthesize 
information  across  topics  rather  than  depending  upon  tests  to  cause  synthesis  to  occur. 

8 »  The  effort  to  increase  the  size  and  comprehensiveness  of  the  test  banks  should 

continue.  Further,  given  the  pressures  on  non-specialist  teachers  to  be  able  to  answer 
inquiries  about  all  aspects  of  the  curriculum,  there  should  be  greater  emphasis  on  the 
development  of  an  expert  system  which  would  identify  the  sources  of  assistance 
available  in  the  course  materials  and  textbooks.  Such  a  system  might  also  extend  the 
time  the  teacher  has  available  to  assist  students  since  it  would  alleviate  some  of  the  more 
routine  student  questions. 

9 .  The  reality  of  the  Departmental  examinations  should  be  recognized  and  provision  made 
to  prepare  students  for  response  formats  which  presently  cannot  be  keyed  into  the  CML 
system. 

10.   Non-print  resources  that  can  be  used  to  augment  the  courses  and  reduce  dependency  on 
print-based  material  should  be  acquired. 
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1 1 .  Since  the  course  materials  must  correspond  to  the  Provincial  Curriculum,  it  would  seem 
advisable  that  there  be  a  close  working  relationship  between  ACS  and  provincial 
curriculum  development  initiatives  not  just  in  the  initial  development  of  a  course  but 
also  when  changes  are  envisaged.  This  would  enhance  the  likelihood  that  the  distance 
learning  materials  would  accurately  reflect  curriculum  intentions  and  that  particular 
requirements  which  impact  on  distance  learners  could  be  addressed. 

1 2.  The  planning  of  appropriate  teaching  strategies  by  distance  learning  teachers  would  be 
enhanced  if  they  received  a  design  overview  for  each  course.  This  should  delineate  for 
each  unit  or  module,the  specific  intentions,  content  knowledge  and  instructional 
processes  which  flow  from  the  Program  of  Studies  and  how  these  relate  to  each  other. 
Such  an  overview  would  provide  branching  options  to  teachers  and  learners.  It  also 
provides  a  student  or  teacher  with  easy  access  to  specific  topics  for  review  or  further 
study. 

13.  Given  the  limited  use  of  audioconferencing,  with  or  without  the  audiographics  system, 
further  consideration  needs  to  be  given  to  assisting  teachers  to  use  these  technologies 
for  effective  instruction.  In  particular,  print  materials  which  would  guide  the  teacher  in 
the  preparation  of  materials,  the  use  of  instructional  strategies,  and  appropriate 
administrative  arrangements  are  needed.  The  necessary  administrative  arrangements 
and  long  distance  costs  may  also  be  limiting  factors  in  the  effective  use  of  this 
technology. 

Teacher  Inservice 

The  transition  from  the  traditional  classroom  to  the  use  of  computer  managed  learning 
and  distance  education  techniques  is  not  necessarily  an  easy  one  for  all  teachers,  and  the 
methods  necessary  to  assist  students  and  manage  classes  in  this  new  environment  are  not 
intuitively  obvious.  Consequentiy  inservice  for  teachers  involved  in  distance  education  should 
be  extended  beyond  training  in  the  use  of  the  equipment  to  include  (1)  pedagogical  techniques 
appropriate  for  different  subject  areas  and ,  (2)  appropriate  configurations  for  distance  learning 
classrooms,  and  (3)  classroom  management  techniques.  Resources  such  as  the  Distance 
Education  Inservice  Kit  developed  for  Alberta  Education  have  been  prepared  to  assist  the 
inservice  education  of  teachers  who  will  be  involved  in  distance  education.  Besides  supporting 
workshops  for  teachers  who  will  be  involved  in  this  mode  of  education.  Alberta  Education 
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should  request  that  the  Faculties  of  Education  in  Alberta  Universities  begin  to  address 
pedagogy  in  distance  education  in  both  their  inservice  and  pre-service  programs. 


Cost  Effectiveness 

Since  no  formal  cost  effectiveness  analysis  was  performed  during  Phase  1,  the 
following  is  offered  as  a  somewhat  terse  analysis  of  costs  for  those  who  may  be  interested  in 
adopting  CML  mathematics.  Except  where  aides  were  engaged  to  assist  with  large  CML 
classes,  the  incremental  costs  associated  with  CML  were  associated  with  the  capital  costs  of  the 
MicroVAX  system  and  the  CBTS  software,  and  the  cost  of  the  student  materials.  All  of  these 
costs  have  changed  since  Phase  L  The  MicroVAX  2000  series  has  been  replaced  by  the  3000 
series.  The  cost  of  the  latter  as  a  stand-alone  system  is  quoted  as  $23,321.30.  The  CML 
system  leased  from  CBTS  is  now  available  to  the  schools  under  a  provincial  license  to  Alberta 
Education.  The  cost  of  the  Distance  Education  student  materials  for  a  5-credit  course  is 
proposed  to  increase  from  $20  per  course  to  $50  per  course.  Assuming  an  amortization  period 
of  five  years  for  the  MicroVAX,  the  annual  cost  of  the  system  would  be  $4,665  plus 
maintenance  costs.  The  present  approach  to  maintenance  appears  to  be  that  of  acquiring  an 
additional  system  for  clusters  of  schools,  and  substituting  parts  from  this  system  for  parts 
which  malfunction  in  the  school  systems.  The  malfunctioning  components  are  then  repaired  by 
Digital  on  a  longer  tum-around,  lower-cost  basis.  Again  amortizing  the  cost  over  five  years, 
this  arrangement  probably  adds  another  $1,000  per  year  to  the  cost  for  each  school.  Finally,  a 
cluster  of  schools  requires  a  coordinator  whose  responsibility  it  is  to  install  equipment,  conduct 
inservice  training,  and  the  like.  In  1989-90,  this  was  estimated  to  cost  $75,000  for  26  schools, 
or  approximately  $2,900  per  school.  This  brings  the  average  cost  per  school  per  year  for 
equipment  and  coordination  to  $  8,565. 

The  per  student  costs  for  the  equipment  decrease  with  an  increase  in  the  number  of 
students,  and  for  supplies  increase  with  an  increase  in  the  number  of  students.  Following  are 
some  sample  figures  expressed  as  cost  per  FTE  Students,  where  an  FTE  student  is  equal  to  35 
credits,  and  5  credits  per  course  is  assumed: 
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Table  4.1 

CML  per  student  costs  for  different  numbers  of  students. 


No.  of  Students 
25 


No.  of  FTE  Students 


Cost 


per  t  lh  Student 
$2726 
1558 


3.6 


50 

75 


10.8 


7.1 


1140 


100 


14.3 


949 


125 


17.9 


827 


The  cost  column  in  this  table  was  derived  by  multiplying  the  number  of  students  by  the  cost  of 
one  5-credit  course  ($50),  adding  the  amortized  capital  costs  plus  maintenance  and  coordination 
costs,  then  dividing  by  the  number  of  FTE  students.  (No  provision  was  made  for  additional 
support  staff  such  as  a  teacher  aide.) 

These  cost  estimates  do  include  the  capital  cost  of  equipment  acquisition.  School 
districts  may  choose  not  to  include  equipment  costs  in  calculating  the  incremental  cost  of  using 
CML.  In  that  case  the  per  FTE  student  costs  would  be  considerably  lower.  However,  school 
districts  should  be  mindful  of  the  fact  that  data  processing  equipment  has  a  relatively  short  life 
expectancy.  They  must  be  prepared  to  upgrade  or  replace  the  system  when  it  no  longer  meets 
the  demands  required  of  it. 

Cost  data  were  not  gathered  for  the  courses  in  which  audioconferencing  took  place^ 
There  do  not,  however,  appear  to  have  been  any  additional  staff  costs  associated  with 
audioconferencing.  At  this  point,  more  needs  to  be  done  to  make  the  audioconferencing  work 
well  before  it  is  appropriate  to  conduct  a  cost  analysis.  Schools  did  note  that  long  distance 
charges  associated  with  audioconferencing  between  some  schools  was  considered  a  significant 
disadvantage  if  the  school  had  to  pay  the  charges. 
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APPENDIX  A 
INTERVIEW  PROTOCOLS 
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PRINCIPAL  INTERVIEW  PROTOCOL  —  DLPN 


1 .  What  elements  of  DL  are  in  your  school?    CML,  TC,  ACS 

2.  From  the  school's  perspective  what,  if  any,  has  been  the  impact  on  .  .  . 

2.1.  ...  student  program 

2.2.  .  .  .  staff 

2.3.  ...  facihties 

2.4.  .  .  .  time-tabling 

2.5.  ...  resources  not  funded  by  Alberta  Education  (e.g.,  libraries) 

2.6.  ...  relationships  to  other  schools  and  districts 

2.7.  .  .  .  school  finances 

2.8.  .  .  .  other? 


3.1  What  assistance  from  the  District  did  you  receive  in  implementing  the  program? 

3.2  What  additional  assistance  do  you  think  you  ought  to  have  received? 


4.  Satisfaction: 

4.1.  What  level  of  satisfaction  do  you  perceive  your  staff  to  have  witii  the  Distance 
Learning  project? 

4.1.1.  What  about  those  not  participating  directly? 

4.2.  How  have  the  parents  of  your  students  reacted  to  the  project? 
4.2. 1   How  have  parents  been  informed  about  the  project? 

4.3.  What  community  reaction  has  there  been  apart  from  parents  of  the  students 
involved? 

4.3.1.  What  indicators  have  you  had? 

4.4.  About  yourself,  how  satisfied  have  you  been  with  the  project  to  date? 

4.4.1.  Has  the  staffing  within  your  school  worked  the  way  you  hoped  it 
would? 

4.4.2.  Can  you  comment  on  the  relationship  between  your  school  and  the  Off- 
site  teachers? 

4.4.3.  How  has  the  project  been  coordinated  in  your  school? 

5.  Expectations: 

5.1.  What  benefits  did  your  school  expect  from  participating  in  the  Distance 
Learning  project? 

5.2.  What  benefits  do  you  feel  your  school  obtained? 

5.3.  What  was  expected  of  your  school  to  participate  in  the  program? 
5.3.1.  Was  your  school  able  to  meet  those  expectations? 

5.4.  What  was  expected  of  you  personally? 

5.4.1.  Were  you  able  to  meet  those  expectations? 
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6.  Tell  me  how  you  went  about  allocating  resources  (e.g.,  staff)  to  the  Distance 
Learning  project  and  why. 

7 .  Let's  talk  about  scheduling  for  audioconferencing  or  live  ACCESS  broadcasts. . . 

8.  Has  communication  with  the  project  leadership  been  adequate  for  you  to 
operate  the  program  satisfactorily? 

8.  L     Do  you  have  suggestions  for  changing  the  nature  of  communication? 

9.  Strengths  &  Weaknesses: 

9.1.  What  would  you  identify  to  be  the  strengths  of  the  project  in  its  present  form? 

9.2.  What  would  you  identify  to  be  the  weaknesses? 

10.  What  would  you  suggest  as  the  best  way  to  organize  and  manage  the  distance  learning 
project  across  the  participating  schools? 

1  L      Generally,  do  you  have  suggestions  for  changes  you  would  like  to  see  in  the 
program? 
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TEACHER  INTERVIEW  PROTOCOL 


Expectations 

1 . 1      What  courses  have  you  been  involved  with  in  DL? 

L2.     Tell  me  generally  what  your  role  has  been  in  the  distance  learning  courses? 

L3.     Now  let's  review  the  details  of  what  you  have  actually  been  doing. 

[  1  ]  Tending  fax 

[  2  ]  Record  keeping 

[  3  ]  Tending  phone 

[  4  ]  Tutoring/teaching  DLSS  subjects 

[  5  ]  Administration 

[  6  ]  Maintenance  of  hardware 

[  7  ]  EMail 

[  8  ]  Computer  Conferencing 

[  9  ]  Computer  Managed  Learning 

[  1  Other 


1 .4      Did  what  you  originally  expected  to  do  differ  from  what  you  have  actually 
done?  [  I  Yes         f  I  No 

IF  YES,  in  what  ways? 

L5      How  were  you  prepared  to  work  with  distance  learning? 

L6.     Have  problems  arisen  in  implementing  the  project  that  you  didn't  expect? 
If  yes,  specify. 


Time 

2.  L  What  have  been  the  demands  on  your  time  as  a  result  of  DL? 
2,2      If  additional  time,  when  is  it  spent? 


Changes  in  Role 

3 .  L     Have  there  been  changes  in  your  role  as  new  technologies  were  introduced? 
3. LI.  EMail  yes  [  ]         no  [  ] 

3.1.2  Computer  Conferencing        yes  [  ]         no  [  ] 

3.1.3  ACCESS  Broadcasts  yes  [  ]  no  [  ] 
3.L4   Computer  Managed  Learning  yes  [  ]         no  [  ] 

3.2.     If  there  have  been  changes,  what  is  the  nature  of  the  change? 


Colleagues 

4.1.     How  functional  has  been  the  relationship  between  you  and  the  off-site 
teachers? 
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4.1.1.  Tell  me  about  some  of  the  things  in  your  relationship  with  the  off- site 
teachers  that  you  were  pleased  with. 

4.1.2.  Tell  me  some  of  the  things  that  you  perceived  as  dysfunctional — didn't 
work  well. 


5 .  Tutoring 

5.1.  How  much  tutoring  took  place  locally? 

5.1.1.  Who  is  doing  the  tutoring?  []  coordinator    [  ]  aide 
[  ]  other  teachers       (  ]  other  students? 

5.L2.  IF  BY  OTHER  TEACHERS,  was  it  []  casual  [J  frequent? 

5 .  L  3 .  IF  BY  OTHER  TEACHERS ,  was  it  [  ]  formal  [  ]  informal? 

5.  L4  IF  BY  OTHER  STUDENTS,  was  it  [  ]  casual  [  )  frequent? 

5.L5   IF  BY  OTHER  STUDENTS,  was  it  [J  formal  [J  informal? 

5.2.  How  frequently  were  the  tutor/markers  called  upon  for  tutoring  assistance? 


6.       ACS  Materials 

6.1.     What  do  you  see  as  the  strengths  of  the  ACS  Materials? 

6.1.1   Old  ACS  Materials 

6.1.1.1     Please  give  examples 


6. 1 .2   New  ACS  Materials 

6.1.2.1      Please  give  examples 


6.2.     What  do  you  see  as  the  weakness  of  the  ACS  Materials? 

6.2.1.  Old  ACS  Materials 

6.2.1.1      Please  give  examples 


6.2.2   New  ACS  Materials 

6.2.2. 1     Please  give  examples 

7 .       Learning/teaching  issues 

7.1  What  has  been  the  impact  on  student  learning,  and  has  it  been  differential? 

7.2  How  has  student  progress  been  tracked  and  managed? 
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8 .       Supporting  resources  (Note  courses  referenced) 

8.1.     How  have  your  physical  facilities  helped  or  hindered  distance  education? 

8.2  Are  there  material  resources  which  you  found  necessary  but  lacking? 
7.2.1   IF  YES,  please  give  examples. 

8.3  Did  you  use  human  resources  other  than  those  formally  available  through  the 
program  (e.g.,  persons  in  the  community)? 

8.4  Were  there  human  resources  for  distance  learning  which  you  would  have  liked 
but  which  were  unavailable  to  you? 

7.4. 1    EF  YES,  please  give  examples. 


9.  Suggestions 

9.1.  What  resources  do  you  feel  need  to  be  added  to  improve  the  program? 

9.2.  What  suggestions  would  you  have  for  improving  the  coordinator  role? 

9.3.  What  other  changes  would  you  suggest? 
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STUDENT  INTERVIEW  PROTOCOL 


1 .  First,  tell  me  which  courses  you  have  been  taking  through  Distance  Learning  this  year. 

2.  Have  you  ever  taken  correspondence  school  courses  before?     [  ]  Yes      [  ]  No 

2A.     IF  YES... 

2. LI.  When? 
2.L2.  What  courses? 

2.  L3.  What  did  you  find  different  between  those  courses  and  the  ones  you  are 
taking  through  Distance  Learning? 


3.       When  did  you  do  most  of  your  work  for  your  DL  courses? 

[  ]  Regularly  scheduled  periods       [  ]   Spares     [  |  Outside  school  hours 


4.       Where  did  you  do  most  of  your  work  for  your  DL  courses? 

[  ]  At  Home     [  ]  DL  Room    [  ]  Library     [  ]   Regular  classroom 


5.       Is  where  you  study  and  when  you  study  for  the  DL  courses  the  same  pattern  as  for 
your  "regular"  courses? 

[  ]  Yes         [  1  No 


6.        How  do  your  grades  compare  between  DL  Courses  and  courses  locally  available  in 
your  school? 

CML  [  1  DL  better  [  ]  DL  the  same       [  ]  DL  poorer 

ACS   [  ]  DL  better  [  J  DL  the  same        [  ]  DL  poorer 

TC      [  ]  DL  better  [  ]  DL  the  same       [  1  DL  poorer 


7.       When  you  were  studying  and  needed  help. .  .how  did  you  get  it? 

[  ]  Didn't  get  help  [  ]  Aide  [  J  ACS  Marker  [  ]  Subject  Teacher 
[  ]  Other  Teachers      [  ]  Other  Students     [  ]  Parents     [  ]  Other 


8.       Which  of  the  following  resources  did  you  use,  if  any? 

Perceived  Usefulness 

Very  Somewhat  Not 

[  ]    hbrary  books                             [  ]  [J  [  ] 

[  ]    films                                       [J  [  ]  [  ] 

[  ]    videotapes/cassettes                     [  ]  [  J  [  ] 

[  ]    filmstrips                                 [  ]  [  ]  [  ] 

[  ]    audiotapes/cassettes                     [  ]  [  ]  [  ] 

[  ]    kits                                         [  ]  [  ]  [  ] 

[  ]    teachers  in  your  school                  [  ]  [  ]  [  ] 

[  ]    Tutor/Markers                            [  ]  [  ]  [  ] 


9.       Let's  talk  about  the  different  components  of  Distance  Learning? 

9.1.    OLD  Correspondence  Course  Materials 

9.1.1.  Was  the  reading  level  [  1  too  easy,  |  |  about  right,  [  1  too  difficult? 

9.1.2.  Was  the  way  the  material  was  organized  [  ]  easy  to  use  or 
[  ]  difficult  to  use? 

9.1.2.1.  Why? 


9.2.    NEW  Correspondence  Course  Materials 

9.2.1.  Was  the  reading  level  []  too  easy,  [  ]  about  right,  [  J  too  difficult? 

9.2.2.  Was  the  way  the  material  was  organized  [  J  easy  to  use  or 
[  ]  difficult  to  use? 

9.1.2.1.  Why? 


9.3.  FAX 

9.3.  L     Was  the  turnaround  dme  getting  feedback  to  you  fast  enough? 
[  ]  Yes      [  ]  No 

9.3.2.     Was  the  feedback  you  received  useful  to  you? 
[  1  Yes      [  ]  No 

9.3.2. 1 .       Did  you  normally  read  the  feedback  you  received? 


9.3.3.     How  was  the  quaUty  of  print  on  the  faxed  materials? 


38 


9.4  Computer 

9.4.1  Did  you  send/received  messages  from  the  Tutor/Marker  using 
electronic  mail?    [  ]  Yes        [  ]  No 

9.4. 1 . 1        How  useful  did  you  find  electronic  mail? 

9.4.2  Did  you  use  the  electronic  bulletin  board? 

1  ]  Yes      [  ]  No 

9.4.2. 1        How  useful  did  you  find  the  electronic  bulletin  board? 


9.5.  Audioconferencing 

9 . 5 .  L    Did  you  participate  in  audioconferencing  sessions? 
[  ]  Yes      [  ]  No 


9.5.1.1.         IFYES.  .. 

9.5.1.1.1.  Which  courses? 

9.5.1.1.2.  Did  you  feel  comfortable  talking? 

IF  NOT,  why  not? 


9.5.1.1.3.    Could  you  easily  understand  what  was  being  said'.^ 
[  ]   Yes         [  1  No 


9.6.    Computer  Managed  Learning 

9.6. 1 .  Did  you  use  a  computer  managed  learning  system? 

I  ]  Yes  [  ]  No 

IFYES... 

9.6.2.  Which  course(s)  did  you  use  it  in? 

9.6.3.  Tell  me  what,  if  anything,  you  liked  about  it. 

9.6.4.  Tell  me  what,  if  anything,  you  disliked  about  it. 


9.6.5  Did  you  find  the  computer  easy  to  use?  [   j  YES    [  J  NO 

9.6.6  What  was  the  role  of  your  teacher  in  the  CML  course(s)? 
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9.7.  Off-site 

9.7J. 


teachers    (ACS  or  another  school) 

Did  you  get  useful  feedback  on  your  assignments? 
[  ]  Yes      [  ]   No         [  ]  Sometimes 


9.7.1.1. 


Give  me  some  examples  that  you  can  think  of. 


9.7.2. 


Did  you  ever  contact  a  tutor  at  ACS  or  in  another  school  directly 
looking  for  assistance?        [  ]  Yes         [  ]  No 


9.7.2.1. 


IF  YES. ..which  of  the  following  did  you  use? 
[  1  phone 
[  J  FAX 

[  ]  electronic  mail 

[  ]  electronic  bulletin  board 


9.7.2.2.  IF  YES. ..did  you  get  the  help  you  needed? 
[  ]  Yes         [  ]  No 

9.7.2.3.  IF  YES.. .were  you  looking  for  help  with  course 
material  or  for  advice  on  other  matters,  for  example, 
what  course  to  take  next? 


4-0^  What  would  you  be  doing  with  your  time  if  Distance  Learning  weren't  available? 

1 1 .  Did  you  find  it  difficult  to  get  assignments  done  on  time?      [  ]  Yes     [  ]  No 
12.1.   IF  YES. ..why? 

12.  For  courses  where  there  wasn't  a  local  teacher  available,  does  it  matter  to  you  that  there 
wasn't  a  local  teacher  for  the  DL  courses  you  took? 

13.  Did  you  find  that  you  needed  materials  that  weren't  available  locally? 

[  ]  Yes         I  ]  No 

13.1.  IF  YES. ..what  kinds  of  material? 

13.2.  Were  you  able  to  get  them?     [  )  Yes        [  ]  No 


14.  What  did  your  school  do  to  prepare  you  to  take  courses  through  Distance  Learning? 

15.  If  you  could  make  changes  in  the  program,  what  changes  would  you  like  to  see? 

16.  Given  your  experience  thus  far  and  choice  in  the  matter,  would  you  continue  to  take 
courses  this  way? 

17.  Would  you  recommend  to  other  students  that  they  take  courses  this  way? 

That's  the  end  of  my  list.  Have  you  got  any  questions  for  me? 
Thanks  very  much  for  helping.  Good  luck. 


13.2.1. 


IF  YES. ..how  and  with  whose  help? 
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QUESTIONS  ADDED  TO  STUDENT  PROTOCOL: 


1.  Did  you  use  the  computer  to  get  your  own  tests? 

2.  Did  you  have  any  difficulty  using  the  computer? 

3 .  Was  the  level  of  difficulty  of  the  tests:  OK?  too  easy? 

4.  Did  you  complete  the  booklet  before  you  took  the  test? 

5 .  What  did  you  do  if  you  did  not  get  80%  on  your  first  try 

Drew  second  test? 
Reviewed  booklet? 
Did  booklet  again? 
Reviewed  test? 

Sought  help — teacher?  peer? 
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